Skip to content

Household calculations and reports apply a dynamic; every simulation refuses another country's; pin API v1 at 003ee4996 - #1113

Draft
anth-volk wants to merge 4 commits into
mainfrom
v2plan/c1-44-household-dynamics
Draft

anth-volk wants to merge 4 commits into
mainfrom
v2plan/c1-44-household-dynamics

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #1112

Household calculations and household reports now apply a dynamic, as economy simulations do. Every simulation now refuses a dynamic stored for another country. The persistence library's pinned API v1 commit moves to 003ee4996 (PolicyEngine/policyengine-api#4325).

What changes

Persistence library

  • Dynamic gains country_id. read_dynamic_reform takes the country and raises UnknownDynamicError for a dynamic stored for another country, as read_policy_reform does for a policy.
  • New read_simulation_reform reads a simulation's policy and dynamic and combines them with combine_reforms, which refuses a parameter both set. It returns a SimulationReform holding the values and the names of the dynamic's parameters. Every reader uses it:
    • economy reads, through read_simulation_values;
    • HouseholdJobStore.read_inputs, which now takes dynamic_id;
    • HouseholdReportStore.read_inputs, which now takes each side's SimulationChoice instead of policy IDs.
  • NewHouseholdJob and HouseholdJobRecord gain dynamic_id, stored in household_jobs.dynamic_id.
  • Household report keys and simulation rows already named each side's dynamic_id; it was always null.
  • public-api-commit moves to 003ee49965a2190e0e5c78f7e946b4bb100c0f16.
  • The test helper insert_dynamic writes dynamics' new required columns instead of name.

Simulation Entrypoint

  • POST /household-calculations takes dynamic_id, and GET /household-calculations/{id} returns it.
  • Each household report side (HouseholdPolicyChoice) takes dynamic_id. A stored household report whose side names a dynamic is now re-run with it, where it used to be refused with 409.
  • New refusals:
    • 400 unknown_dynamic, also for a dynamic of another country;
    • 422 missing_parameters at dynamic_id, for a dynamic setting parameters the catalog version lacks;
    • 422 overlapping_parameters at dynamic_id, for a dynamic setting a parameter the policy also sets.
  • A value the household contract refuses is located at dynamic_id when the dynamic sets its parameter.
  • Economy requests and household reports build these refusals with one function, simulation_values_refusal. The unknown_dynamic detail now names the country.

The v1 run_simulation code imports nothing this adds. API v1 sends no household dynamic_id until C1 · 45.

Tests

  • Red first: 1717adda committed the failing tests. Then 29 of 50 persistence cases and 40 of 54 entry cases failed, for the expected reasons.
  • On the final head, every added or changed test passes by node ID: 57 persistence cases, 95 entry cases and 4 generated-client cases.
  • The persistence cases include the five comparisons with API v1's tables. They ran against a disposable PostgreSQL 17 database that API v1's chain built at 003ee4996, with its privilege manifest applied.
  • The generated-client cases ran against a client built from this branch by scripts/generate-clients.sh.
  • Ruff, Pyright (persistence, entry, apis-integ) and the quality guards pass.
  • On unchanged main, all five comparisons also pass at 003ee4996. Between 63cce8273 and 003ee4996, API v1's tables changed only in dynamics and in tables sim doesn't model.

Deploy order

Production must have API v1's revision 9d1d12a7ac58 before this releases.

  • With this change, sim reads dynamics.country_id whenever a request, or a stored record being re-run, names a dynamic.
  • Without that column, such a request fails with a database error, answered 503 database_unavailable, where it is answered 400 today. Requests without a dynamic don't read dynamics, and no readiness check reads the column.
  • A merge to main releases and deploys by itself, so this PR should merge only after API v1's production release containing #4325.
  • Order: API v1's release with #4325 in production, then this PR, then C1 · 45 (API v1 sends household dynamics).

Read from GitHub on 2026-11-20:

  • API v1's release commit 80a42ddeb, which follows 003ee4996, has a Push run still marked in progress.
  • Its Seed staging v2 database job succeeded, so staging has the revision. Upgrade staging v1 Cloud SQL schema failed.
  • No production job has run, so production doesn't have the revision yet.

Grants

From API v1's privileges.py at 003ee4996, no new grant is needed:

  • dynamics: table-level SELECT for the simulation service role and the Modal worker role.
  • household_jobs: SELECT, INSERT for the service role, so it can insert dynamic_id, plus UPDATE on the call-start columns.
  • simulations, parameter_values, parameters, policies, households and the catalog: unchanged.

The persistence tests ran as policyengine_v2_simulation_service on a database with those grants applied. Sim's readiness list (SERVICE_TABLE_PRIVILEGES) already requires SELECT on dynamics.

Pin order with #1111, and overlap

Review

  • Round 1 found two existing tests that the change broke, and both are fixed:
    • a route case that still expected a side's dynamic_id to be refused;
    • a generated-client test that read a job state without the now-required dynamic_id.
  • Round 2 found no bugs. Its quality findings are fixed: the shared refusal function, SimulationReform passed through, and documentation.

🤖 Generated with Claude Code

anth-volk and others added 4 commits November 20, 2026 01:15
… country (failing)

Household jobs take and store a dynamic_id, household reports and their
re-runs take one per side, and each merges the dynamic's values into the
household reform, as economy simulations do. Every simulation refuses a
dynamic stored for another country. The insert_dynamic test helper writes
the columns API v1's revision 9d1d12a7ac58 requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ther country's

Household jobs take a dynamic_id, store it in household_jobs.dynamic_id and
return it in the job's state; household reports and their re-runs take one
per side. Each reads the policy's and the dynamic's values together with the
new read_simulation_reform, as economy simulations now do too, so a
parameter both set is refused. The household report key already names each
side's dynamic, and a stored household report naming one is now re-run with
it instead of refused.

The Dynamic model gains country_id, and read_dynamic_reform refuses a
dynamic stored for another country as UnknownDynamicError, which every
route answers as unknown_dynamic. A value the household contract refuses is
located at dynamic_id when the dynamic sets its parameter.

public-api-commit moves to 003ee4996, API v1's merge of PolicyEngine/policyengine-api#4325, whose
revision 9d1d12a7ac58 gives dynamics their country.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, and require dynamic_parameters

Review round 1 found two existing tests that the change broke: a route case
still expected a household report side's dynamic_id to be refused, and the
generated-client test read a household job state without the now-required
dynamic_id. Both now expect the new shapes, and new route and generated-client
tests send each side's dynamic and read it back.

The inputs that carry which parameters a dynamic sets now require them, so
no caller can leave them out and have a refused dynamic value located at
policy_id. The README paragraphs this change edited are rewrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h its dynamic's parameters

Review round 2 found no bugs. Its quality findings:
- Economy requests and household reports now build the refusal of a
  simulation's policy or dynamic with one function,
  simulation_values_refusal, so each says the same thing and locates it the
  same way. The household calculation route keeps its own refusals, whose
  details and error types differ.
- The household inputs carry each side's SimulationReform, the combined
  values with the parameters its dynamic sets, instead of two parallel
  fields.
- The entry README's re-run refusals and the CallerRowRefusalCode comment
  name the dynamic refusals, and two README paragraphs are rewrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Household calculations and household reports apply a dynamic, and every simulation refuses another country's dynamic

1 participant